Bandit and covariate processes, with finite or non-denumerable set of arms

We introduce herein a new approach to nonparametric multi-armed bandit theory involving both the and covariate processes. Following Berry et al. (1997), we assume non-denumerable set of arms for process. The develop can be readily extended continuous-time processes by using ?-greedy randomization arm elimination instead dynamic allocation indices. It also carries out stochastic search with O(1) expected time nearly optimal at values in given B before applying elimination. procedure is shown attain asymptotically minimal rates regret over B.